Reinforcement Learning-Based Optimal Battery Control Under Cycle-Based Degradation Cost

Battery energy storage systems are providing increasing level of benefits to power grid operations by decreasing the resource uncertainty and supporting frequency regulation. Thus, it is crucial obtain optimal policy for utility-level battery efficiently provide these grid-services while accounting its degradation cost. To solve control (OBC) problem using powerful reinforcement learning (RL) algorithms, this paper aims develop a new representation cycle-based model according rainflow algorithm. As latter depends on full trajectory, existing work has rely linearized approximation converting into instantaneous terms Markov Decision Process (MDP) based formulation. We propose MDP form introducing additional state variables keep track past switching points determining cycle depth. The proposed allows adopt deep Q-Network (DQN) RL algorithm search OBC policy. Numerical tests real market data have demonstrated performance improvements in enhancing mitigating degradation, as compared earlier approximation.